Verifying File System Consistency at Runtime

Existing file-system reliability methods, such as checksums, redundancy, or transactional updates, provide limited defenses against file-system bugs that cause disk corruption. The existing workarounds, based on using backups or repairing the file system, are painfully slow. Worse, the recovery is performed much after the error occurred, and thus may result in further corruption and data loss. We present a system that protects file system metadata from buggy file system operations. Our approach leverages modern file systems that provide crash consistency using transactional updates. We define declarative statements called "consistency invariants" for a file system. These invariants must be satisfied by each transaction being committed to disk to preserve file system integrity. By checking each transaction before it commits, we can minimize the damage caused by buggy file systems. The major challenges to this approach are specifying invariants, and correctly interpreting file system behaviour without relying on the file system code. Our prototype system, called Recon, provides a framework for file-system specific metadata interpretation and invariant checking. We show the feasibility of interpreting metadata and writing consistency invariants for the Linux ext3 and btrfs file systems in this framework. For ext3, Recon can detect random as well as targeted file-system corruption at runtime as effectively as the offline e2fsck filesystem checker, with low performance overhead.